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CLAIMS 

What is claimed is 

1. A method of automatically managing a plurality of remote workers carrying out a 
variety of jobs for one or more customers, each job including a process of a set of one or 
more task steps and a set of units of source data, the method comprising: 

storing in a database information on each remote worker including one or 
more task skills of the worker that define the types of task steps the worker is 
certified to carry out; 

storing in the database information on the customers; 

storing in the database information on each process, including the customer of 
the process, the order of carrying out the task steps of the process, how the input 
for each task step is obtained from the results of prior task steps in the process, 
and any pre-processing and post-processing required; 

receiving the units of source data from the customers; 

carrying out any defined pre-processing for the received source data; 

storing in a task data structure information on tasks to be completed, each task 
defined by a task step and a unit of input for the task step; 

storing in the database information on each remote worker including one or 
more tasks skills of the worker that define the types of task steps the worker is 
certified to carry out; 

receiving requests from one or more of the remote workers for tasks; 

upon receiving a task request from a remote worker, dispatching a task from 
the stored tasks to be completed to the remote worker according to one or more 
task dispatch rules; 
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receiving the task result from the remote workers for the task dispatched to the 
workers; 

carrying out any defined post-processing of the task result corresponding to 
the tasks of a process for a unit of source data to produce result data for the unit 
of source data; 

sending the result data to the customers; 

managing the capacity of the system based on information about the stored 
tasks, 

assessing the quality of at least some of the task results; and 

evaluating the workers who carried out the tasks that produced the task results 
whose quality is assessed, 

such that the quality assessing and the worker evaluating occur automatically 
substantially without human management. 

A method as recited in claim 1, wherein the variety of jobs include a plurality of 
members of the set consisting of: data entry, telesales, voice transcription, translation, 
image categorization, sales lead incubation, auditing, repair of documents after OCR, 
photo retouching, paralegal processes, call center quality assurance, and editorial work. 

A method as recited in claim 1, wherein the assessing the quality of at least some of 
the task results includes determining which task results of which tasks should undergo 
quality assessment. 

A method as recited in claim 3, wherein the information stored in the database for at 
least some task steps includes a default QA sampling rate for the task step, and wherein 
the determining which task results should undergo quality assessment depends on the 
default QA sampling rate for the task step that produced the task result. 
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A method as recited in claim 1, wherein the quality assessing of a particular task 
result includes creating a QA task having the particular task result as input, the QA task 
being a new task for the method to be dispatched to a QA worker, the QA worker being 
a worker having a task skill to carry out the task step of the QA task. 

A system for automatically managing a plurality of remote workersxarrying out a 
variety of jobs for one or more customers, each job including a process of a set of one or 
more task steps and a set of associated source data units, the system connected to a 
network, each worker having one or more task skills and able to communicate with the 
system using a worker terminal connectable to network, the system comprising: 

a storage subsystem containing a task data structure to store tasks to be 
completed, each task defined by a task step and a unit of input for the task step; 

a pre-processor coupled to the storage subsystem to accept units of source data 
from the customers and to carry out any defined pre-processing for the accepted 
source data; 

a task dispatcher coupled to the network and to the task data structure to 
accept requests from one or more of the remote workers for tasks and to dispatch 
a task from the task data structure to a remote worker requesting tasks, the 
dispatching according to one or more task dispatch rules; 

a task submission unit coupled to network to receive the task result from the 
remote workers for the task dispatched to the workers; 

a capacity manager coupled to the storage subsystem to manage the capacity 
of the system based on task load information on the tasks in the task data 
structure, on the available workers, and on the available worker task skills; 

a quality unit coupled to the task submission unit and having the task results as 
input to assess the quality of at least some of the task results; 

a certification unit coupled to the dispatcher to certify workers as having one 
or more task skills; 



47 



an evaluation unit coupled to the storage subsystem to evaluate the workers 
who carried out the tasks that produced the task results whose quality is assessed; 
and 

a post-processor coupled to the network and to the quality unit to produce 
result data from the task results corresponding to the tasks of a process for a unit 
of source data, including any defined post-processing of the task results, and to 
send the result data to the customer of the process, 

such that the quality unit assesses the quality of the task results and the evaluation 
unit evaluates workers automatically substantially without human management. 

A system as recited in claim 6, 

wherein the storage subsystem further includes 

a database storing information on each remote worker including one or more 
task skills of the worker that define the types of task steps the worker is certified 
to carry out, information on one or more customers, and information on each 
process, the process information including the customer of the process, the order 
of carrying out the task steps of the process, how the input for each task step is 
obtained from the results of prior task steps in the process, and any pre- 
processing and post-processing required; and 

a data store for storing input and output information for the tasks, and 

wherein the coupling between the certification unit and the task dispatcher, the capacity 
manager, and the evaluation unit is via the database. 

A system as recited in claim 6, wherein the variety of jobs include a plurality of 
members of the set consisting of: data entry, telesales, voice transcription, translation, 
image categorization, sales lead incubation, auditing, repair of documents after OCR, 
photo retouching, paralegal processes, call center quality assurance, and editorial work. 
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A system for automatically managing a plurality of remote workers carrying out a 
variety of jobs for one or more customers, each job including a process of a set of one or 
more task steps and a set of associated source data units, the system connected to a 
network, each worker able to communicate with the system using a worker terminal 
connectable to network, the system comprising: 

a storage subsystem containing 

a database storing information on each remote worker including one or 
more tasks skills of the worker that define the types of task steps the 
worker is certified to carry out, information on one or more customers, 
and information on each process, the process information including the 
customer of the process, the order of carrying out the task steps of the 
process, how the input for each task step is obtained from the results of 
prior task steps in the process, and any pre-processing and post- 
processing required; 

a task data structure to store tasks to be completed, each task defined 
by a task step and a unit of input for the task step; and 

a data store for storing input and output information for the tasks; 

a mechanism coupled to the storage subsystem to accept units of source data 
from the customers; 

a pre-processor coupled to the storage subsystem to carry out any defined pre- 
processing for the accepted source data; 

a mechanism coupled to the network to accept requests from one or more of 
the remote workers for tasks; 

a task dispatcher coupled to the storage subsystem and to the network for 
dispatching a task from the task data structure to a remote worker requesting 
tasks, the dispatching according to one or more task dispatch rules; 
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a task submission unit coupled to the storage subsystem to receive the task 
result from the remote workers for the task dispatched to the workers; 

a post-processor coupled to the storage subsystem to carry out any defined 
post-processing of the task results corresponding to the tasks of a process for a 
unit of source data to produce result data for the unit of source data; 

a mechanism coupled to the storage subsystem to send the result data to the 
customers; 

a capacity manager coupled to the storage subsystem to manage the capacity 
of the system based on task load information on the tasks in the task data 
structure, on the available workers, and on the available worker task skills; 

a quality unit coupled to the storage subsystem to assess the quality of at least 
some of the task results; and 

an evaluation unit coupled to the storage subsystem to evaluate the workers 
who carried out the tasks that produced the task results whose quality is assessed, 

such that the quality unit assesses the quality of task results and the evaluation unit 
evaluates workers automatically substantially without human management. 

A system as recited in claim 9, wherein the variety of jobs include a plurality of 
members of the set consisting of: data entry, telesales, voice transcription, translation, 
image categorization, sales lead incubation, auditing, repair of documents after OCR, 
photo retouching, paralegal processes, call center quality assurance, and editorial work. 

A method of automatically managing a plurality of workers^arrying out a variety of 
jobs for one or more customers, each job including a process of a set of one or more task 
steps and a set of units of source data, the method comprising: 

storing in a database information on each worker including one or more task 
skills of the worker that define the types of task steps the worker is certified to 
carry out; 
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storing in the database information on each process; 
receiving the units of source data; 

storing in a task data structure information on tasks to be completed, each task 
defined by a task step and input for the task step; 

storing in the database information on each remote worker including one or 
more task skills of the worker that define the types of task steps the worker is 
certified to carry out; 

dispatching a task from the stored tasks to be completed to a worker; 

receiving the task result from the worker for the task dispatched to the worker 
after the worker completes the task; and 

assessing the quality of at least some of the task results, and 

wherein the quality assessing occurs automatically substantially without human 
management. 

A method as recited in claim 11, further comprising: 

producing result data from the task results of the tasks of a process and 
sending the result data to the customer of the process;. 

A method as recited in claim 1 1, wherein dispatching is to a remote worker via the 
Internet and wherein the remote worker completes the task at remote location. 

A method as recited in claim 13, wherein the storing of process information includes 
storing information on any required pre-processing of source data and on any required 
post-processing, and wherein the source data receiving includes carrying out any pre- 
processing required for the source data according to the stored process information, and 
wherein the producing result data further includes carrying out any post-processing 
required according to the stored process information. 
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15. A method as recited in claim 11, wherein the dispatching occurs upon receiving a task 
request from the worker. 

16. A method as recited in claim 11, wherein the task request is received from the worker 
automatically when the worker logs on. 

17. A method as recited in claim 1 1, wherein the variety of jobs include a plurality of 
members of the set consisting of: data entry, telesales, voice transcription, translation, 
image categorization, sales lead incubation, auditing, repair of documents after OCR, 
photo retouching, paralegal processes, call center quality assurance, and editorial work. 

18. A method as recited in claim 11, further comprising: 

certifying workers as having one or more task skills. 

19. A method as recited in claim 18, wherein the dispatching occurs according to a set of 
one or more dispatch rules. 

20. A method as recited in claim 19, wherein the dispatch rules includes that the worker a 
task is assigned to must have the task skill for the task step. 

21. A method as recited in claim 19, wherein the dispatching further occurs to satisfy one 
or more task dispatch objectives. 

22. A method as recited in claim 19, wherein the task data structure is part of the database 
and wherein the dispatching includes forming a query on the database. 

23. A method as recited in claim 22, wherein the database is a relational database 
including a set of tables. 

24. A method as recited in claim 18, wherein the assessing the quality of at least some of 
the task results includes determining which task results of which tasks should undergo 
quality assessment. 
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25. A method as recited in claim 24, wherein the information stored in the database for 
the skill levels of each worker includes a skill quality level, and wherein the determining 
which task results should undergo quality assessment depends on the skill quality level 
of the remote worker that produced the task result. 

26. A method as recited in claim 24, wherein the information stored in the database for 
each remote worker includes a worker status, including whether or not the remote 
worker is on probation, and wherein the determining which task results should undergo 
quality assessment depends on whether or not the remote worker that produced the task 
result is on probation. 

27. A method as recited in claim 24, wherein the information stored in the database for at 
least some task steps includes a default QA sampling rate for the task step, and wherein 
the determining which task results should undergo quality assessment depends on the 
default QA sampling rate for the task step that produced the task result. 

28. A method as recited in claim 18, wherein the quality assessing of a particular task 
result includes creating a QA task having the particular task result as input, the QA task 
being a new task for the method to be dispatched to a QA worker, the Q A worker being 
a worker having a task skill to carry out the task step of the QA task. 

29. A method as recited in claim 28, wherein QA tasks are created for at least some of the 
task results according to a sampling rate for the task step that produced the particular 
task result. 

30. A method as recited in claim 18, further comprising 

evaluating the workers who carried out the tasks that produced the task results 
whose quality is assessed. 

31. A method as recited in claim 30, wherein the information stored in the database for 
the skill levels of each worker includes a skill quality level. 
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32. A method as recited in claim 30, wherein the information stored in the database for 
each worker includes a worker status, including whether or not the worker is on 
probation. 

33. A method as recited in claim 32, wherein the evaluating of a worker depends on the 
status of the worker. 

34. A method as recited in claim 33, wherein the information stored in the database for 
the skill levels of each worker includes a skill quality level, and wherein the evaluation 
of the worker may have an effect on one or both of the status of the worker and of the 
skill quality level of the worker. 

35. A system for automatically managing a plurality of worker&xarrying out a variety of 



jobs for one or more customers, each job including a process of a set of one or more task 
steps and a set of associated source data units, the system connected to a network, each 
workers having one or more task skills and able to communicate with the system using a 
worker terminal connectable to network, the system comprising: 

a storage subsystem containing a task data structure to store tasks to be 
completed, each task defined by a task step and input for the task step from 
source data received from the customer; 

a task dispatcher coupled to the network and to the task data structure to 
dispatch a task from the task data structure to an available worker; 

a task submission unit coupled to network to receive the task result from the 
worker for the task dispatched to the worker; 

a quality unit having the task results as input to assess the quality of at least 
some of the task results; and 

such that the quality unit assesses the quality of at least some of the task results 
automatically substantially without human management. 

36. A system as recited in claim 35, further comprising: 




54 

a certification unit coupled to the dispatcher to certify workers as having one 
or more task skills. 

A system as recited in claim 36, wherein the recruitment unit further is to screen 
potential workers, each successfully screened potential worker becoming an applicant. 

A system as recited in claim 37, wherein the training unit further is to train applicants, 
and wherein the certification unit further is to certify applicants as having one or more 
task skills. 

A system as recited in claim 36, 

wherein the storage subsystem further includes 

a database storing information on each remote worker including one or more 
task skills of the worker that define the types of task steps the worker is certified 
to carry out, information on one or more customers, and information on each 
process, the process information including the customer of the process, the order 
of carrying out the task steps of the process, how the input for each task step is 
obtained from the results of prior task steps in the process, and any pre- 
processing and post-processing required; and 

a data store for storing input and output information for the tasks. 

A system as recited in claim 35, wherein dispatching is to a remote worker via the 
Internet and wherein the remote worker completes the task at remote location. 

A system as recited in claim 35, wherein the dispatching occurs upon receiving a task 
request from the worker. 

A system as recited in claim 35, wherein the task request is received from the worker 
automatically when the worker logs on to the system. 
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43. A system as recited in claim 35, wherein the variety of jobs include a plurality of 
members of the set consisting of: data entry, telesales, voice transcription, translation, 
image categorization, sales lead incubation, auditing, repair of documents after OCR, 
photo retouching, paralegal processes, call center quality assurance, and editorial work. 

44. A system as recited in claim 39, wherein the task data structure is part of the database. 

45. A system as recited in claim 39, wherein the data store is part of the database. 

46. A system as recited in claim 39, wherein the database is a relational database 
including a set of tables. 

47. A system as recited in claim 39, wherein the quality unit is further to determine which 
task results of which tasks should undergo quality assessment. 

48. A system as recited in claim 47, wherein the quality unit is coupled to the database, 
wherein information stored in the database for the skill levels of each worker includes a 
skill quality level, and wherein the determining which task results should undergo 
quality assessment depends on the skill quality level of the remote worker that produced 
the task result. 

49. A system as recited in claim 47, wherein the quality unit is coupled to the database, 
wherein the information stored in the database for each remote worker includes a worker 
status, including whether or not the remote worker is on probation, and wherein the 
determining which task results should undergo quality assessment depends on whether 
or not the remote worker that produced the task result is on probation. 

50. A system as recited in claim 47, wherein the quality unit is coupled to the database, 
wherein the information stored in the database for at least some task steps includes a 
default QA sampling rate for the task step, and wherein the determining which task 
results should undergo quality assessment depends on the default QA sampling rate for 
the task step that produced the task result. 
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51. A system as recited in claim 39, wherein the quality unit is coupled to the task 
dispatcher, and wherein the quality assessing of a particular task result includes creating 
a QA task having the particular task result as input, the QA task being a new task for the 
task dispatcher to be dispatched to a QA worker, the QA worker being a worker having a 
task skill to carry out the task step of the QA task. 

52. A system as recited in claim 51, wherein the quality unit is coupled to the database, 
and wherein QA tasks are created for at least some of the task results according to a 
sampling rate for the task step that produced the particular task result. 

53. A system as recited in claim 39, further comprising 

an evaluation unit coupled to the quality unit and to the database to evaluate 
the workers who carried out the tasks that produced the task results whose quality 
is assessed. 

54. A system as recited in claim 53, wherein the information stored in the database for the 
skill levels of each worker includes a skill quality level. 

55. A system as recited in claim 53, wherein the information stored in the database for 
each worker includes a worker status, including whether or not the worker is on 
probation. 

56. A system as recited in claim 55, wherein the evaluating of a worker depends on the 
status of the worker. 

57. A system as recited in claim 56, wherein the information stored in the database for the 
skill levels of each worker includes a skill quality level, and wherein the evaluation of 
the worker may have an effect on one or both of the status of the worker and of the skill 
quality level of the worker. 

58. A system for automatically managing a plurality of workei^carrying out a variety of 
jobs for one or more customers, each job including a process of a set of one or more task 
steps and a set of units of source data, the method comprising: 

a storage means containing: 
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a database for storing information on each process and information on 
each worker including one or more task skills of the worker that define 
the types of task steps the worker is certified to carry out, and 

a task data structure for storing information on tasks to be completed, 
each task defined by a task step and input for the task step; 

means for receiving the units of source data; 

means for dispatching a task from the stored tasks to be completed to a 
worker; 

means for receiving the task result from the worker for the task dispatched to 
the worker after the worker completes the task; 

means for producing result data from the task results of the tasks of a process 
and sending the result data to the customer of the process; 

means for assessing the quality of at least some of the task results; and 

means for certifying workers as having one or more task skills. 

59. A system as recited in claim 58, wherein the system is coupled to the Internet and 
wherein the dispatching means dispatches to a remote worker via the Internet and 
wherein the remote worker completes the task at remote location. 

60. A system as recited in claim 58, wherein the variety of jobs include a plurality of 
members of the set consisting of: data entry, telesales, voice transcription, translation, 
image categorization, sales lead incubation, auditing, repair of documents after OCR, 
photo retouching, paralegal processes, call center quality assurance, and editorial work. 

61. A system as recited in claim 58, wherein the assessing the quality of at least some of 
the task results includes determining which task results of which tasks should undergo 
quality assessment. 
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62. A system as recited in claim 61, wherein the information stored in the database for at 
least some task steps includes a default QA sampling rate for the task step, and wherein 
the determining which task results should undergo quality assessment depends on the 
default QA sampling rate for the task step that produced the task result. 

63. A system as recited in claim 58, wherein the quality assessing of a particular task 
result includes creating a QA task having the particular task result as input, the QA task 
being a new task for the method to be dispatched to a QA worker, the QA worker being 
a worker having a task skill to carry out the task step of the QA task. 

64. A system as recited in claim 58, further comprising 

means for evaluating the workers who carried out the tasks that produced the 
task results whose quality is assessed. 

65. A system as recited in claim 64, wherein the information stored in the database for the 
skill levels of each worker includes a skill quality level and wherein the evaluation of the 
worker may have an effect on the skill quality level of the worker. 

66. A system as recited in claim 64, wherein the information stored in the database for 
each worker includes a worker status, including whether or not the worker is on 
probation, and wherein the evaluating of a worker depends on the status of the worker. 

67. A carrier medium carrying computer readable code segments to instruct one or more 
processors of a processing system to carry out a method of automatically managing a 
plurality of workers^arrying out a variety or jobs for one or more customers, each job 
including a process of a set of one or more task steps and a set of units of source data, 
the medium comprising: 

one or more code segments to instruct the one or more processors to store in a 
database information on each remote worker and on each process, the worker 
information including one or more task skills of the worker that define the types 
of task steps the worker is certified to carry out; 
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one or more code segments to instruct the one or more processors to store in a 
task data structure information on tasks to be completed, each task defined by a 
task step and input for the task step corresponding to source data from the 
customer of the process of the task step; 

one or more code segments to instruct the one or more processors to dispatch a 
task from the stored tasks to be completed to a worker; 

one or more code segments to instruct the one or more processors to accept 
task result from the worker for the task dispatched to the worker; 

one or more code segments to instruct the one or more processors to send 
result data to the customers; 

one or more code segments to instruct the one or more processors to assess the 
quality of at least some of the task results; and 

one or more code segments to instruct the one or more processors to certify 
applicants and workers as having one or more task skills. 

A carrier medium as recited in claim 67, wherein dispatching is to a remote worker 
via the Internet and wherein the remote worker completes the task at remote location. 

A carrier medium as recited in claim 67, wherein the variety of jobs include a 
plurality of members of the set consisting of: data entry, telesales, voice transcription, 
translation, image categorization, sales lead incubation, auditing, repair of documents 
after OCR, photo retouching, paralegal processes, call center quality assurance, and 
editorial work. 

A carrier medium as recited in claim 67, wherein the assessing the quality of at least 
some of the task results includes determining which task result of which tasks should 
undergo quality assessment. 
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71. A carrier medium as recited in claim 67, wherein the quality assessing of a particular 
task result includes creating a QA task having the particular task result as input, the QA 
task being a new task for the method to be dispatched to a QA worker, the QA worker 
being a worker having a task skill to carry out the task step of the QA task. 

72. A carrier medium as recited in claim 67, further comprising 

one or more code segments to instruct the one or more processors to evaluate 
the workers who carried out the tasks that produced the task results whose quality 
is assessed. 

73. A carrier medium as recited in claim 72, wherein the information stored in the 
database for the skill levels of each worker includes a skill quality level and wherein the 
evaluation of the worker may have an effect on the skill quality level of the worker. 



74. A carrier medium as recited in claim 72, wherein the information stored in the 

database for each worker includes a worker status, including whether or not the worker 
is on probation, and wherein the evaluating of a worker depends on the status of the 



75. A computer implemented method of automatically managing one or more human 



to produce result data in electronic form, the process including a set of one or more task 
steps, each tasl^pJiaving an mput^congspo nding to thej ourc^data and when 
completed on the input resulting in a corresponding task result, the method comprising: 

receiving units of source data from a customer; 

for each unit of source data 
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worker. 
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for each taskjtgp of the set for the unit of source data; 

dispatching the t ask step and its corresponding, input unit to a 
worker; and 
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receiving from the worker, after the worker carries out the 
dispatched task step on the input unit, the task result 
corresponding to the dispatched task step and input unit, 



wherein each worker is certified to have one or more task skills, wherein each task step 
requires a corresponding task skill, and wherein the dispatching of any task step occurs 
automaticall y substantiail y^ithout.hurnan intervention to a worker who is certified to 



the method further com prising ass essing at lea st some of the task results of at teast some 
of task steps for at least some of the units of source data. 

76. A method as recited in claim 75, wherein the task step dispatching is from a server 
computer system over a network to a remote worker, and wherein the worker carries out 
the task step at a location remote from the server computer system. 

77. A method as recited in claim 75, wherein the source data and the result data is 
provided in electronic form. 

78. A method as recited in claim 75, further comprising, for each unit of source data, 



have the corresponding task sKill oHKelaskstep, 




generating the result data for the unit of source data from one or more of the 
task results corresponding to the task steps of the set; and 



sending the result data for the unit of source data to the customer. 



79. 



A system as recited in claim 35, further comprising: 



a post-processor coupled to the network and to the quality unit to produce 
result data from the task results of the tasks a process and to send the result data 
to the customer of the process. 



